MODELING UTTERANCE-DRIVEN VISUAL ATTENTION DURING SITUATED COMPREHENSION
نویسندگان
چکیده
منابع مشابه
Modeling Utterance-driven Visual Attention during Situated Comprehension
Evidence from behavioral studies demonstrates that spoken language guides attention in a related visual scene and that attended scene information can influence the comprehension process. Here we model sentence comprehension within visual contexts. A recurrent neural network is trained to associate the linguistic input with the visual scene and to produce the interpretation of the described even...
متن کاملModeling Utterance-mediated Attention in Situated Language Comprehension
Empirical evidence from studies using the visual world paradigm reveals that spoken language guides attention in a related visual scene and that scene information can influence the comprehension process. Here we model sentence comprehension using the visual context. A recurrent neural network is trained to associate the linguistic input with the visual scene and to produce the interpretation of...
متن کاملSpeaker gaze affects utterance comprehension beyond visual attention shifts
Previous research has shown that listeners follow speaker gaze to mentioned objects in a shared visual environment to ground referring expressions, both for human and robot speakers. What is less clear is whether listeners exploit speaker gaze to infer referential intentions (Staudte & Crocker, 2010), or whether the benefits of gaze can be more simply explained by (reflexive) gaze following (Fr...
متن کاملVisual Attention during Spatial Language Comprehension
Spatial terms such as "above", "in front of", and "on the left of" are all essential for describing the location of one object relative to another object in everyday communication. Apprehending such spatial relations involves relating linguistic to object representations by means of attention. This requires at least one attentional shift, and models such as the Attentional Vector Sum (AVS) pred...
متن کاملModeling the Directionality of Attention During Spatial Language Comprehension
It is known that the comprehension of spatial prepositions involves the deployment of visual attention. For example, consider the sentence “The salt is to the left of the stove”. Researchers [29,30] have theorized that people must shift their attention from the stove (the reference object, RO) to the salt (the located object, LO) in order to comprehend the sentence. Such a shift was also implic...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neural Network World
سال: 2012
ISSN: 1210-0552,2336-4335
DOI: 10.14311/nnw.2012.22.006